The target size of the Unmanned Aerial Vehicle （UAV） is small， and the characteristics among multiple UAVs are not obvious. At the same time， the interference of birds and flying insects brings a huge challenge to the accurate detection and stable tracking of the UAV targets. Aiming at the problem of poor detection performance and unstable tracking of small target UAVs by using traditional target detection algorithms， a real-time tracking algorithm for multiple UAVs based on improved PaddlePaddle-YOLO （PP-YOLO） and Simple Online and Realtime Tracking with a Deep association metric （Deep-SORT） was proposed. Firstly， the squeeze-excitation module was integrated into PP-YOLO detection algorithm to achieve feature extraction and detection of UAV targets. Secondly， the Mish activation function was introduced into ResNet50-vd structure to solve the problem of vanishing gradient in the back propagation process and further improve the detection precision. Thirdly， Deep-SORT algorithm was used to track UAV targets in real time， and the backbone network that extracts appearance features was replaced with ResNet50， thereby improving the original network’s weak perceptual ability of small appearances. Finally， the loss function Margin Loss was introduced， which not only improved the class separability， but also strengthened the tightness within the class and the difference between classes. Experimental results show that the detection mean Average Precision （mAP） of the proposed algorithm is increased by 2.27 percentage points compared to that of the original PP-YOLO algorithm， and the tracking accuracy of the proposed algorithm is increased by 4.5 percentage points compared to that of the original Deep-SORT algorithm. The proposed algorithm has a tracking accuracy of 91.6%， can track multiple UAV targets within 600 m in real time， and effectively solves the problem of "frame loss" in the tracking process.